A block-based transform approach has been the primary choice for transforms in current video compression schemes and standards, such as the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group-4 (MPEG-4) Part 10 Advanced Video Coding (AVC) Standard/International Telecommunication Union, Telecommunication Sector (ITU-T) H.264 Recommendation (hereinafter the “MPEG-4 AVC Standard”), when compared with more advanced transform approaches (such as sub-band coding, for example) due to its inherently lower complexity and achievement of comparable performance. Lapped transforms perform significantly better than non-overlapping transforms such as discrete cosine transforms (DCTs) while incurring just a small increase in complexity. Lapped transforms can be designed to maximize coding gain, maximize energy compaction, provide good frequency response, maximize regularity in the basis, or maximize a combination of the above objectives. The coding gain is especially of interest since it translates directly to an improvement in the rate-distortion performance. The coding gain of a transform is computed as the ratio of the “reconstruction distortion without transform” to that of the “reconstruction distortion with transform”. Under the high-bitrate assumption, this quantity for a lapped bi-orthogonal transform (LBT) is described in a first prior art approach as follows:
                              G          TC                =                              {                                          ∏                                  i                  =                  1                                M                            ⁢                                                          ⁢                              [                                                      (                                                                  σ                                                  y                          i                                                2                                                                    σ                        x                        2                                                              )                                    ⁢                                                                                                          P                        i                                                  -                          1                                                                                                            2                                                  ]                                      }                                -                          1              M                                                          (        1        )            where σx2 is the variance of source x, y is the output of the lapped transform, σyi2 is the variance of the ith transform output, and Pi−1 is the ith synthesis basis (post-filter column) of the lapped transform. The design of a high bitrate lapped transform requires that the coding gain defined in Equation (1) is maximized.
In a second prior art approach, an alternate equivalent is disclosed for decomposing the quasi-optimal lapped transform into a pre-filter operation followed by a shifted DCT operation. The advantage is that the pre-filter approach can be applied outside of existing encoder and decoder loops, therefore the second prior art approach requires little change within existing encoders and decoders.
For the pre-filtering based approach to lapped transform, the output y can be represented as follows:y=DCT[Shift(Px)]  (2)where Shift is a time-shift between the pre-filter and the block transform, and P is the pre-filter applied on current data x.
Turning to FIG. 1, an implementation of a 4×8 lapped transform as a 4×4 pre-filter followed by a shifted 4×4 DCT operation is indicated generally by the reference numeral 100 That is, FIG. 1 depicts two equivalent implementations of the lapped transform. In the top portion of FIG. 1, 8 input samples are directly transformed into 4 output samples by the lapped bi-orthogonal transform. Note that in order to have the same number of total input and output samples, the next 8 input samples are taken with an overlap with the first 8 input samples, as can be observed in FIG. 1. Regarding the bottom portion of FIG. 1, where the implementation uses a pre-filter, the following is involved: first a pre-filter is applied to 4 input samples; and then a DCT is applied. Note that the shift between the pre-filter and the discrete cosine transform allows for the processing of 8 input samples for each 4 output samples in exactly the same way as the top portion of FIG. 1.
Previous efforts to augment the block-based coding approach such as that performed in the MPEG-4 AVC Standard include using pre-processing filters and post-processing filters and increasing the coding gain while ignoring the impact on predictive-coding efficiency. However, such prior art pre-filters are designed to work with only a single transform. For example, a third prior art approach involves a scheme in which the 4×4 pre-filter was designed to work with the 4×4 DCT for intra-coding only. Additionally, the third prior art approach does not give any consideration to modifying dependent encoder and decoder blocks such as the rate-distortion optimizer and the most probable mode predictor to work in unison with pre-filtering and post-filtering and achieve a higher compression efficiency.